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Description 

BIT-RATE CONTROL METHOD AND APPARATUS FOR 
NORMALIZING VISUAL QUALITY 
Technical Field 

[1 ] The present Invention relates to video coding. More particularly, the present 

invention relates to a method and an apparatus for controlling hitrates by use of in- 
formation available to a pre-decoder so as to .minimize the peak signal-to-noise ratio 
(PSNR) varianeein a wavelet-based scalable video coding using the pre-decoder. 
Background Art 

[2] Scalable video ceding (allowing partial decoding at various resolutions, qualities 

and temporal levels from a single compressed bitstrearn) is widely considered a 
promising technology for efficient signal representation and transmission in het- 
erogeneous en vironments. Although MPEG-4 Fine Granularity Scalability (FGS) is es- 
tablished as a signal-to-noise ratio (SNR) and temporal scalable video coding standard 
many wavelet-based scalable video ceding schemes have already demonstrated their 
potential for SNR, spatial, and temporal scalability. Detailed information on MPEG- 
4FG5 may be obtained from a report published by Mr. W. Li, 'Overview of fine 
granularity scalability in MPEG-4 video standard/( IEEE Trans. Circuits Syst Video 
TechnoL, vol. II, pp. 301-317, Mar. 2001.). 

[3] FIG. 1 is a block diagram illustrating an. overall configuration of a video codec 

based on a conventional rate-distortion (R-D) optimization art. The video codec 100 
includes a rate control module 130 that chooses an optimal quantization step or an 
amount of optimal bits for each coding unit, an encoder 1 10 that generates a bit-stream 
40 whose bandwidth is limited and a decoder 120 that reconstructs image-sequences 
20 from the bandwidth-limited bit-stream 40. In the conventional art, the rate-control is 
only performed in the encoder 1 10. 

[4] FIG. 2 is a block diagram illustrating an operational configuration of a wavelet- 

based scalable video codec according to the conventional art. 

[5] Although rate control algorithms generally improve R-D performance, the con- 

ventional methods all utilize prediction error information that is only usable in the 
encoding phase, which implies that the rate control should be done in the encoder 210. 
For most applications that require fully scalable video coders, the encoder 21 0 should 
generate a sufficiently large bit stream 35 such that a pre-decoder or transcodbr 220 
extracts an adequate amount of bits 40 from the bit stream while considering quality, 
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temporal, and spatial requirements. The conditions for extracting art appropriate 
amount of bit-stream consistent with quality, temporal and spatial requirements are 
referred to as scalability conditions. Then, a decoder 230 can recover a video sequence 
20 from the truncated bit stream 40. 

[6] The rate control should be date in a pre-decoder 220 instead of the encoder 

because the actual bit-rate is determined in the pie-decoder 220. There has been little 
research on rate control algorithms in the pre-decoder 220, and most research has 
focused on a constant bit-rate (CBR) scheme. However, Mr. Hsiang suggests a 
variable bit-rate (VBR) scheme in his PhD dissertation, 'Highly scalable subbandV 
wavelet image and video ceding, '(Rensselaer Polytechnic Institute, New York, Jan, 
2002.),' which can also be used in a pre-decoder (hereinafter referred as 'Hsiang's 
scheme'). In this scheme, wavelet bit planes used in the pre-decoder are identical in 
number in order to enhance performance of the conventional CBR scheme. 

[71 Hereinbeiow, Hsiang's scheme will be described in detail. 

\ 81 In the following description, the transmitted video can be partitioned into multiple 

group-of-pictures (GOP), with each GOP having multiple frames. This can simplify a 
rate allocation algorithm because each GOP is separately encoded Thus, each GOP is 
independent from one another, however, each frame in a GOP is heavily correlated 
with one another. If B f is the total bits for an entire video sequence that consists of N 
GOPs, the rate-allocation problem can be formulated as 

m 

[1.0] where R(i) is the allocated bits for the i-' GOP and D(i) is absolute difference 

between original and decoded frames. A fundamental aspect of the VBR method is to 
allocate more bits to relatively complex scene* and less bits to the others in order to 
achieve better R-D performance or visual quality If we define scene complesty as the 
degree of difficulty for encoding the given image frame, the amount of allocated bits 
for a COP, with a constant number of used wavelet bitplanes, is highly correlated with 
the relative scene eorapleaty among GOPs. From this fact, Hsiang's scheme proposes 
that the VBR scheme equalize the number of bitplanes used for all the frames. 

[1 1] If b(i, j) is the number of encoded bits for the i- " COP and the j-* b) (plane and B(i, 

k) represents the number of accumulated encoded bits using k bitplanes, then B(i, k) is 
defined as 
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r.i2] 

[13] If the number of bitplanes used is a constant value K for all the frames, then B(i, 

K) gives some statistics of scene complexity for the i- * frame with the total allocated 
bite, A(K), given by 

[14] 

M^)^T^B(iX) Formula.! 

f 1 5] where N is total number of GOPs . If K* represents an integer number of bitplanes 

whose total amount of allocated bits is closest to B , the final allocated bits for the ith 
GOP, R (i), can be given by 

[16] 

R ( Xi) ~ B(L K ) Formula 4 

[17] where 
[18] 

A(K-\)<B T <A(K) 

[19] By using a linear interpolation technique, it may be possible to obtain more 

accurate statistics of scene complexity by making the total encoded bits equal to B ^ 
Disclosure of Invention 

Technical Problem 

[20] Wavelet-based scalable vide coding inherently employs the property of embedding, 

and thus, it is appropriate to use it in a variable bit-rate (VBR) algorithm. On this 
point, although Hsiang's scheme is simple and effective, it needs further improvement 
in order to reduce the variation of PSNR values since it focuses merely to minimize the 
objective error measure. Even if the average PSNR is sufficiently high, noticeable 
visual artifacts can be observed in the low PSNR frames if the PSNR variance is high. 
Therefore, it is valuable to have a hit allocation scheme that minimizes the PSNR 
variance. 

Technical Solution 

[21] In view of the above, a method lot i] ui mi u formation , sUL m a 

pre-decoder side is provided so as to allow a decoder side to have an optimal quality. 
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[22] A method of allocating variable bit-rates is also provided so as to minimize PSNR 

variance in the wavelet-based scalable video eating. 

[23] According to an aspect of the present invention, there is provided a bit-rate control 

method comprising, a first step of determining an amount of bits for each eoting unit 
from a btt-stream generated by encoding an original moving picture, so as to allow a 
visual quality of the moving picture to be uniform relative to the coding units thereof; 
and a second step of extracting a bit-stream having the amount of bits as desired by 
truncating apart of the bit-stream based on the determined bit amount, 

[24] According to another aspect of the present invention, there is provided a bit-rate 

control apparatus comprising, a first means for determining a bit amount for each 
coding unit from a bit-stream generated by encoding an original moving picture, so as 
to make the visual quality of the moving picture uniform relative to the coding unit 
thereof, and a second means for extracti ng a bit-stream having the amount of bits as 
desired by truncating a part of the bit-stream based on the determined bit amount. 
Description of Drawings 

[25] The above and other objects, features and advantages of the present invention will 

be reaSIy apparent from the following detailed description of exemplary emboimenis 
when taken in conjunction with the accompanying drawings, in which: 

[26] FIG. 1 is a block diagram illustrating an overall configuration of a video codec 

based on the conventional rate -distortion optimization ait; 

[27] FIG. 2 is a block diagram illustrating an operational configuration of a wavelet- 

based scalable video codec according to the conventional art: 

[28] FIG. 3 is a block diagram illustrating an operational configuration of a wavelet- 

based scalable video codec according to an exemplary embedment of the present 
invention; 

[29] HG. 4 is a graph illustrating a comparison of D(i)/D and B(i, K*) in an encoded 

Canoa QCiF(Quarter Common Interchange format) sequence; 
[30] FIG. 5 is a graph illustrating a bit-rate allocated for each GOP in a Football QCIF 

sequence; 

[3 1 ] FIG. 6 is a graph illustrating an average PSNR for each GOP in a Football QOF 

sequence; 

[32] FIGs. 7 and 8 illustrate examples of the 92-* frame of a Foreman QCIF sequence 

coded to VBR-D and VBR-N, respectively; and 
[33] FIGs. 9 and 10 illustrate examples of the 106-* frame of a Foreman QCIF sequence 

coded to VBR-D and VBR-N, respectively. 
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Mode for Invention 

[34] Hereinafter, an exemplary embodiment of the present invention will be described 

in detail with reference to the accompanying drawings. 

[36] FIG. 3 is a block diagram illustrating at) operational configuration of a wavelet- 

based scalable video codec according to an exemplar}' embodiment of the present 
invention, 

[36] A scalable video codec 300 includes an encoder 310 that encodes an original 

moving picture 10 so as to generate a sufficiently targe bit-stream 2? a rate control 
unit 340 that allocates the optimal amount of bits for each coding unit based on a bit- 
rate 30 desired by a user; a pre-decoder 320 that receives the bit-stream 35 and extracts 
a bit-stream 40 having an appropriate amount of bits by truncating apart of the 
recei ved bit-stream: 35, based on the optimal amount of bits selected in the rate control 
unit 340; and a decoder 330 that decodes image sequences of the moving picture from 
the extracted bit-stream 40, so as to reconstruct the original moving picture. 

\ 37] In particular, the present invention focuses on the operation performed in the rate 

control unit 340. The rate control unit 340comprises tour steps and operates a 
definition step of a bit-rate function available for use in the predecoder 320 by using a 
bit distribution and a distortion function with a constant number of bitplanes, a pro- 
summation step of the bit-rate by moJfyng the bit-rate function to thereby obtain the 
uniform visual quality, an approximation step of the distortion function by use of the 
bit distribution to determine the distortion function, and a normalization step of the 
modified bit-rate function to allow the total allocated bit-rates to be equal to a target 
bit-rate. Because the assessed visual quality of a picture is generally based on the 
PSNR, PSNR is also employed in the present invention as a criterion for quality 
assessment. Additionally, Mean Absolute Distribution (MAD) information, used in the 
conventional encoder, is replaced with bit dstribution of the constant number of 
bitplanes as a scene complexity function. 

[38] The definition step of a bit-rate function available for use in the predecoderby 

using a bit distribution and a distortion function with a constant number of bitplanes 
will be described Similar to Formula 6, let us assume that the source statistics are 
Laplacian distributed 

[39] 

■^(•0 ~~e * • Foroudafc 
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where « is a constant 

If a dt fference function is used as a distortion measure, then there is a closed form 
solution for the rate distortion function as derived in Formula 7. D(i) denotes a 
distortion function, indicating a difference between the original image and the final 
image after decompression. 



ML- 

Mif) " 



The R-D function can be further modified by introducing two new parameters: 
MAD and nontexture overhead Formula 8, 



#(/)-//{/) _ ln f .1 \ 

M{1) [ami)) Formula* 

In Formula 8, H(i) denotes the bits used for header information and motion vectors, 
and M(i) denotes the MAD computed using motion-compensated residual for a 
luminance component. MAD is included in an R-D function in order to consider scene 
complexity since moa^ bits should he used for relatively complex frames and less bits 
for others at the sane target bit-rate limitation. 

Although the conventional VBR scheme uses B(i, K*) as the allocated bits, the 
present invention uses B(i, K*) to replace M(i) since B(i, K*) is highly correlated with 
the scene complexity for ith GOP. By replacing M(i) with B(i, K*), the following is 
yielded 



Rii) J J ) 

For notations! simplicity, the nontexture overhead, H{i) is not considered in 
Formula 9 and the remaining text of this description since it is a trivial problem. In the 
inventors' preliminary experiments, it has been shown that, by choosing the optimal 
value of a, this replacement is reasonable for many combinations of bit-rates, 
resolution, and sequences. 

The prc~summation step of the bit-rate obtains the uniform visual quality by 
modifying the bit-rate function and will now be described. 

If D is the average value of D(i) for all GOPs, then atfitg In (D(i) / D) to both 
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sides of Formula 9 gives 



B(iX) 




Formula Hi 



where 




formula i l 



Since the right skle of Formula 1 0 is a constant value, it follows that allocating 
R'(i) bits for i-th GOP results in a constant distortion. To obtain R'(i), R{i) and In 
(D(i)/D) should be computed as shown in Formula 1 1 . However, this may be a difficult 
problem since the actual distortion D(i) cannot be determined in the pre-decoder. 

The approbation step of the distortion function by use of the bit distribution to 
determine the distortion function will now be described 

To solve the above problem, the initial bit allocation R(i) is first set equal to R (i) 
as described above, and D(i)/D is estimated by some approximations. In Formula 11 , 
D(i)/D is the ratio of the relative magnitude of distortion to the average distortion. 
Because a relative magnitude of distortion increases when the scene complexity does, 
it is assumed that D(i)/D can be represented in terms of the scene complexly function, 



and r is m experimental constant used to compensate for the nonlinearity between 
the actual distortion and the allocated bits, F JG. 4 shows the comparison graph of 
D0VD and B(i, K*)/B in Canoa QCIF sequence encoded at 512 fcbps with the value of 
r=0,4. As shown in FIG. 4, D(i)/D, can be roughly modeled by the relative scene 
complexly, B(i, K*) ! / B. Furthermore, from the exhaustive preliminary experiments, 



B(i, K*), as 



^ BjiXY 
D * B 



where 




FeomsLi U 
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it has been shown that the value of rM).4 is satisfactory for almost ail the tt 
conditions. 

[61] Inserting Formula 12 to Formula 1! fields 

[62] 



1651 



NB(i,K*Y 



} B(j t K*) r j Formula I- 



[63] The normalization step of the modified bit-rate function to allow the total allocated 

bit-rates to be equal to a target bit-rate will now be described 
[64] Since R'(i} is modified from R(i) without considering fee bit-rate limitation, R'(i) 

should be normalized to meet the target bit-rate requirement. Simple normalization 

gives a final equation defined as 



[66] where R (i) in the allocated bits for i-th GOP, which can flatten the distortion. 

[67] CBR indicates the con ventional scheme for constant bit-rate allocation, VBR-0 

indicates variable rate allocation according to Hsiang's scheme, and VBR-N indicates 
variable rate allocation accoiding to the present invention. As shown in Table 1, the 
VBR-N scheme outperforms the CBR scheme's Foreman OC1F and Canoa OC3F by a 
clear margin up to 0.9 dB and 0.6 dB, respectively, due to VBR-N scheme's efficient 
realization of adaptive bit allocation technique. In addition all performance gaps 
between the VBR-D and the VBR-N are limited within about 0.2 dB for both 



[68] 
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Table 1 



Bit-rate (kbps) 


CBR 


VBR-D 


VBR-N 


Foreman QCIF@30Hz 


64 


27.5? 


27.98 


27.80 


128 


32.30 


32.93 


32.71 


256 


36.40 


37.05 


36.90 


384 


38.91 


39.40 


39.31 


512 


40.73 


41.21 


41.17 


768 


43.63 


43.97 


43.91 


Canoa QCIF@30Hz 


64 


23.43 


23.59 


23.54 


128 


26.34 


26.48 


26.41 


256 


29.26 


29.42 


29.40 


384 


31.39 


31.53 


31.50 


512 


33.27 


33.44 


33.40 


768 


36.31 


36.48 


36.46 



[69] Table 2 shows the standard devi atkm of PSNR values using CBR, VBR-D, and 

VBR-N. First, this table reveals that VBR-D and VBR-N schemes reduce the PSNR 
standard deviation more than the CBR scheme. In the standard deviation of PSNR 
obtained per each frame, VBR-N reduces it by 23% to 50.8% in comparison with 
VBR-D, although it has not expressly been shown. Since VBR-N employs an op- 
timization technique based on COP, the percentage of reduction becomes very large, in 
the standard deviation of PSNR obtained by each GOP, so called, GOP-average PSNR 
standard deviation. This demonstrates that VBR-N scheme is more effective in making 
the overall PSNR curve flat. Referring to Table 2, VBR-N reduces GOP-average 
PSNR standard de viation by 26. 1 % to 89.7% in comparison with VBR-D. 



Tabic 2 



Bit-rate (kbps) 


CBR 


VBR-D 


VBR-N 


1-VBR-N/VBR~D<%) 


Foreman QCIF@30Hz 


64 


1.93 


1.51 


0.73 


51.7 


128 


2.44 


1.92 


LOO 


47.7 


256 


2.33 


1.69 


0.48 


71.3 


384 


2,06 


1.34 


0.26 


80.9 


512 


1.89 


1.19 


0.25 


79.4 


768 


1.61 


0.97 


0.32 


67.5 


Canoa ot 11 a JOB; 


64 


1.29 


1.10 


0.81 


26.1 


128 


1.23 


0.98 


0.50 


49.1 


256 


1.22 


0.88 


0.23 


74.0 


384 


.1.17 


0.75 


0.08 


89.7 


512 


1.14 


0.76 


0.10 


87.4 


768 


1.12 


0.69 


0.21 


69.2 
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[71] FIG. 5 is a graph illustrating a bit-rate allocated for each GOP in a Football QCIF 

sequence, and 6 is a graph illustrating an average PSNR for each GOP in a Football 
QCIF sequence. Football QCIF is encoded at art average bit-rate of 512 kbps. 
Moreover, we illustrate GOP-averaged PSNR instead of frame PSNR to investigate the 
overall flatness of the PSNR curve. In FIG. 5, the bit-rates of CBR are almost constant 
and those of VBR-D and VBR-N are highly variable since they are optimized by scene 
characteristics, which are highly variable. On the other side, the GOP-averaged PSNR 
curve of VBR-N is much flatter than that of CBR and VBR-D. 

[72] FIGS. 7, 8, 9 and 1 0 illustrate several examples of coding Foreman QCIF 

sequences. 

[73] FIG. 7 illustrates the 92- * frame (PSNR-38.02) generated by VBR-D and FIG. 8 

illustrates the 92-* frame (PSNR=39.94) generated by VBR-N on the same position. 

[74] As shown in these figures, VBR-N reduces an artifact significantly. It is a natural 

result since VBR-N can flatten the PSNR curve with a slightly smaller average PSNR, 
thus, the minimum value of PSNR increases significantly 

[751 FIG. 9 illustrates the 106-* frame (PSNR=44.05) generated by VBR-D and FIG. 10 

illustrates the 106-* frame (PSNR=44X)2) generated by VBR-N, 

[76] As shown in these figures, although the PSNR value of VBR-D is higher than that 

of VBR-N, the actual visual quality is almost the same because both PSNR values are 
high enough to make coding artifacts imperceptible. This property is very useful for 
subjective visual quality because the visual quality can be controlled in a more 
perceptual sense by improving the PSNR of poor quality frames by sacrificing that of 
very good quality frames. 

Industrial Applicability 

[77] According to the present invention, the PSNR standard deviation may be greatly 

reduced while maintaining almost the average PSNR as it is. This property is very 
useful for subjective visual quality because the visual quality can be controlled in a 
more perceptual sense by improving the PSNR of poor quality frames by sacrificing 
that of very good quality frames. 

[78] According to the present invention, since information available only on the pre- 

deeoder side is used the pre-decoder needs no additional information. 

[79] Although the present invention has been described in connection with the preferred 

embodiment of the present in vention, it will be apparent to those skilled in the art that 
various modifications and changes may be made thereto without departing from the 
scope and spirit of the invention. Therefore, it should be understood that the above 
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embodiment is not restrictive but illustrative in all aspects. The scope of the present 
invention is defined by the appended claims rather than the detailed description of the 
invention. All modifications and changes derived from the scope and spirit of the 
claims and equivalents thereof should be construed to be inchiied in the scope of the 
present invention. 



wo TMSflmm 



12 



PCT/K.R20O4/002623 



Claims 

[13 A bit-rate control method comprising; 

determining an amount of bits for each of a plurality of ceding units from a bit- 
stream generated by encoding an original moving picture, so as to allow a visual 
quality of the moving picture to be uniform relative to the coding units thereof; 
and 

extracting a bit-stream having the amount of bits by truncating a part of the bit- 
stream based on the determined bit amount. 

[2] The method as claimed in claim 1 , wherein a peak signal-to-noise ratio (PSNR) 

is employed as a reference for measuring the visual quality. 

[3] The method as claimed in claim 1 f wherein the bitstream generated by an 

encoder follows a wavelet-based video coding scheme and is modified 
adaptiveiy to a scalability condition by the pre-decoder, 

[4] The method as claimed in claim 1 , wherein a flattening of a quality measuring 

reference is performed by increasing the bits allocated for a first coding unit and 
decreasing the bits allocated for a second coding unit, and 
wherein the first coding unit has a lower quality image than the second coding 
unit. 

[5] The method as claimed in claim 1, wherein the detercmnation of the bit amount 

includes: 

defining a bit-rate function available in the pre-decoder by using a bit dis- 
tribution and a distortion function with a constant number of bitplanes; and 
presuming the bit-rate by modifying the bit-rate function, so as to obtain uniform 
visual quality. 

f 6] The method as claimed in claim 5, wherein the determination of the bit amount 

further includes initially approximating the distortion function, with the use of 
the bit distribution, to determine the distortion function with information useable 
in the pre-decoder. 

[7] The method as claimed in claim 6, wherein the determination of the bit amount 

further includes normalizing the bit-rate function by modifying the bit-rate 
function so that a total allocated bit-rate is equal to a target bit-rate. 

[8] A bit-rate control apparatus, comprising; 

a first means for determining a bit amount for each of a plurality of coding unite 
from a bit-stream generated by encoding an original moving picture, so as to 
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make the visual quality of the moving picture uniform relative to the ceding units 



a second means for extracting a bit-stream having the amount of bits by 
truncating a part of the bit-stream based on the detercnraed bit amount. 

[9] The apparatus as claimed in claim 8 t wherein the bitstream created by an 

encoder, which follows a wavelet-based video coding scheme, is modified 
adaptively to a scalability condition by the pre-decoder. 

[10] The apparatus as claimed in claim 8, wherein the first means includes a means 

for defining a bit-rate function available in the pre-decoder by using a bit d 
istritatfoa and a distortion function widi a constant number of bitplanes, and 
presuming the bit-rate by modifying the bit-rate function, so as to obtain uniform 
visual quality. 

[11] The apparatus as claimed in claim 1.0, wherein the first, means further includes a 

means for initially approximating the distortion function, with the use of the bit 
distribution, to determine the distortion function with information useable in the 



[12] The apparatus as claimed in claim 1.0, wherein the first means further includes a 

means for normalizing the bit-rate function by modifying the bit-rate function so 

that a total allocated bit-rate equals a target bit-rate. 
[1 3] A computer-readable recording medium for recording a computer program code 

for enabling a computer to provide a service of a bit-rate control method 

comprising; 

determining an amount of bits for each of a plurality of coding units from, a bit- 
stream generated by encoding an original moving picture, so as to allow a visual 
quality of the moving picture to be uniform relative to the coding units thereof; 
and 

extracting a bit-stream having the amount of bits by truncating a part of the bit- 
stream based on the determined bit amount, 
f 1 4] A bit-rate control apparatus, comprising: 

a determining unit determining a bit amount for each of a plurality of coding 
units from a bit-stream generated by encoding an original moving picture, so as 
to make the visual quality of the moving picture uni form relative to the coding 



an extracting unit extracting a bit-stream having the amount of bits by truncating 
a part of the bit-stream based on the determined bit amount. 
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1 1 5] The apparatus as claimed in claim 1.4, wherein the bitstroam created by an 

encoder follows a wavelet-based video coding scheme and is modified 
afaptively to a scalability condition by the pre-decoder. 

[1 6] The apparatus as claimed in claim 14, wherein the determining unit includes a 

defining unit defining a bit-rate function available in the pre-decoder by using a 
bit distribution and a distortion function with a constant number of bitplanes, and 
a pro-summation unit modifying the bit-rate function so as to obtain uniform 
visual quality. 

[1 7] The apparatus as claimed in claim 16, wherein the determining unit further 

includes an approximating unit that initially approximates the distortion function, 
with the use of the bit distribution, in order to determine the distortion function 
with information useable in the pre-decoder, 

[18] The apparatus as claimed in claim ! 6, wherein the determining unit further 

includes a normalizing unit normalizing the bit-rate function by modifying the 
bit-rate function so that a total allocated bit-rate equals a target bit-rate. 
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